RESEARCH PAPERS 


EXAMINING THE EFFICACY OF PROJECT-BASED LEARNING 
ON CULTIVATING THE 21 ST CENTURY SKILLS AMONG HIGH 
SCHOOL STUDENTS IN A GLOBAL CONTEXT 

By 

CHI-SYAN LIN * JUNG-TSAN MA ** KAREN YI-CHWEN KUO *** 

CHIEN-TZU CANDACE CHOU **** 

* Professor, Department of Information and Learning Technology at the National University of Tainan, Taiwan. 

** Doctoral Student in National University of Tainan, Taiwan. 

*** Faculty Member, Tainan University of Technology, Taiwan. 

**** Associate Professor of Learning Technology, University of St. Thomas, Minneapolis, Minnesota, United States. 

ABSTRACT 

The goal of the study is to explore the opportunities and challenges associated with Project Based Learning strategy in a 
global context on the aspects of both fostering learning community of practices and nurturing the 21 st century skills. For 
collecting empirical data, the study implements and administers an online international project-based learning 
program for high school students. Since it is claimed that interpersonal interaction is key to the success of online projecf- 
based learning programs, the study first focuses on investigating patterns of community of practice, which is a 
combination of online learning behaviors and interpersonal interaction, among students against a claimed theoretical 
framework. For unveiling the impacts on tertilizing the 21 s 'century skills, the study examines the perceptions of students 
and teachers participating in the online international project-based learning program. Results of the study confirm the 
asserted theoretical framework regarding the community of practices and reveal that students of various counfries 
perform differently in terms of community of practices in virtual learning space. As to building the 21 s ' century skills on 
students, the study credits the project-based learning strategy deploying in a global context for providing intuitive and 
essential aspects of learning ingredients to students tor pursuing skills related to Communication, Collaboration, 
Complex Problem-solving, Critical Thinking, and Creativity. 

Keywords: Community of Practice, Virtual Learning Environments, Project Based Learning, International Collaborative 
Learning, Online Learning Behaviors. 


INTRODUCTION 

There is a widespread belief that Information and 
Communication Technology (ICT) can and will empower 
teachers and learners, transforming teaching and learning 
paradigms from being highly teacher-dominated to 
student-centered, and that this transformation will result in 
increased learning gains for students, creating and 
allowing opportunities for learners to develop their 
creativity, problem-solving abilities, collaboration skills, 
communication skills, and critical thinking skills. However, 
there are currently very limited data to support this belief. 

Furthermore, a study conducted by Organization for 
Economic Co-operation and Development (OECD, 2011), 
examines how students' access to and use ICT over the 


years and explores how different National Education 
systems are integrating ICT into students' learning 
experiences and claims that educators have not yet 
become good enough at the kind of pedagogies that 
make the most of technology even in advanced countries. 
With the full adoption of ICT in sight in schools, educators 
are therefore under pressure to implement and 
demonstrate effective pedagogical practices. 

1. Literature Review 

Project Based Learning (PBL) is a premium pedagogy that 
has four main components: (1) a realistic problem, (2) 
structured group work, (3) multi-faceted assessment, and 
(4) participation in a professional learning network (Buck 
Institute for Education, 2009; Moursund, 2003). On the other 
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hand, it has been claimed that the 21 st century skills such as 
critical thinking, problem-solving skills, communication 
skills, collaboration skills, and creativity are essential for the 
modern citizenship (Reigeluth & Karnopp, 2013). PBL, in the 
mean times, has been considered long as a significant 
student-centered approach to engage students and 
cultivate the 21st century skills on them (OECD, 2011; 
Reigeluth & Karnopp, 2013). Whereas, it has been 
proposed that the best practice of PBL is in networked 
learning environments energized with learning community 
(Lin&Ma, 2011). 

Indeed, more and more results of research reveal that 
interpersonal communication among learners can lead to 
an enhanced performance for all in the scenario of 
networked learning (McFarlane, 2006; Di Bias & Poggi, 
2008). By fostering interaction among people who use 
them, networked learning environments indeed support 
deep learning and high level of engagement in practice. 
Bronack et al. (2006) asserted that networked learning must 
account for the social nature of learning. They claim that 
learners are compelled through actions as members of a 
community in networked learning environments. The 
growth and development of learners lie at the confluence 
of effort and reflection, within the context of meaningful 
scenario and activity, as individual learners become 
increasingly central participants within the community. It 
implies that participation within a community of practice 
prompts both more- and less-experienced members to 
engage in reflective thinking and complex problem¬ 
solving. It is a widely accepted claim that by fostering 
interaction among experts, peers, learning objects, and 
activities in formal, informal, and serendipitous ways, a 
networked learning environment embedded with 
community of practice can help learners attain their 
constructive knowledge (meta knowledge), instead of 
cognitive knowledge alone, Of the many tools that can be 
used to facilitate collaborative learning, Tarsiero (2007) 
asserted that community of practice is promising, 
especially in addressing unstructured learning tasks that are 
impossible to foresee and too complex to formalize such 
as the PBL. Nevertheless, the fostering of learning 
community and meaningful communication taking place 
in it cannot be taken for granted; examples of unsuccessful 


attempts to encourage learners to incorporate online 
communication to support their learning are common 
even where the online discussion among learners is a 
compulsory element of a credit bearing online course 
(McFarlane, 2006). 

Ultimately, building up and sustaining vibrant online learning 
communities in a networked learning environment is 
imperative to engage learners in learning process in that it 
can provide an unbounded social as well as fantastic space 
for learners, which is essential to the claims of constructive 
learning theory (Lin & Kuo, 2006). As a matter of facts, online 
learning communities serve as virtual atmospheres or 
ambiances that are unparalleled to any existing motivator 
for engaging interpersonal interactions and are the cores 
of networked learning environments. The interpersonal 
interactions such as negotiation, communication, and 
collaboration that happen inside online learning 
community could ignite a learner to exchange or reflect his 
or her newly acquired knowledge with members of the 
communities, and contribute it into a shared knowledge 
pool. Tarsiero (2007) also pointed out that online learning 
Communities Of Practice (COP) tends to involve a higher 
number of diverse individuals from many regions and 
countries allowing access to a broader expertise with far 
lower costs. The facts above imply that the nature of fantasy 
and versatility in online learning communities might inject 
the profound human content into and enrich networked 
learning environments (Friedl, 2003). However, there are 
few evidences or knowledge so far about how school 
students behave in the virtual learning environment in terms 
of learning behaviors and interpersonal interaction. 

As to the relationship between PBL and cultivating the 21 st 
century skills, a recent study reveals that teachers who use 
PBL report more gains of 21 st century skills, compared with a 
closely matched comparison group (Boss, et al., 2011). 
That means students in PBL classrooms are spending more 
time learning about important content through 
experiences that emphasize critical thinking, collaboration, 
creativity, and communication (Ravitz, et al., 2012). 
However, it is still waiting for answers about whether the 
combination of PBL and learning community of practice in 
a global context might better fostering the 21 st century skills 
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or not. 

2. Methodology and Procedure of the Study 
2.7 Theoretical Framework 

It is asserted that there are five hierarchical levels, which are 
Observing, Participatory, Contributory, Collective, and 
Pervasive, in terms of aspects of interactivity and 
engagement of online users in a virtual space (Operand, 
2009). In the beginning of showing up in a virtual space, first 
and foremost, people usually are simply browsing around 
and have no control over the selection of content or the 
pacing of exploration. After a while, people will start to join 
the online activity and manipulate elements within the 
activity. When users are getting familiar with the setting, they 
are enlisted to contribute original content and information. 
This is just what the Web 2.0 is about. But up to this point, 
people only pursue interaction with platform or digital 
content. The next level of interactivity or engagement, 
which is collective level, starts to involve with the aspect of 
human interaction. In this level, the diversity of knowledge 
and viewpoints from a large community can be distilled 
into a collective wisdom. At the highest level of interactivity 
and engagement, the interactive experience will become 
pervasive, which means that online human interaction 
may seamlessly be embedded into the daily lives of its 
members of the community. The aforementioned five 
hierarchical levels of interactivity and engagement are 
aligned with what Ito (2009) claimed that people would 
advance from "hanging out" to "messing around", and 
then to reach the phase of "geeking out" in terms of social 
networking with Information Technology. The study will 
explore patterns of learners 1 online behaviors and 
engagement following these categories of immersion in a 
virtual space. Figure 1 depicts the theoretical framework of 
the online behaviors adopted by the study, 


Immersion levels in Terms of COP 

o 


| Hanging Out 


► I Messing Around~| ~ 


\ Geeking Out 


KobseiMng|2^participJitoiy|3^contributo^|4^collecUv<^^^5^I)e^a^Il 



Figure 1. The Theoretical Framework of the Online Behaviors 


In 2011, OECD announced the result of the first PISA 
(Programme for International Student Assessment) survey of 
digital literacy (OECD, 2011). The result indicates that 
performance of the kids around the globe over the use of 
computer and the Internet to learn are different across 
countries. In the list of the result, Korea, New Zealand, and 
Australia are on the top. Meanwhile, the result of the OECD 
survey prompts a further question: Do the kids around the 
globe also behave differently in the virtual space in terms of 
their interpersonal interaction? 

The comparative studies of global PISA conducted by 
OECD are receiving attention increasingly across countries 
and among professionals. The results of the study might 
verify the results of the PISA and provide better insights 
about the difference on the performance among involved 
countries in terms of networked learning behaviors. 
Furthermore, the special feature with its global context of 
the study could deliberately provide multiple aspects of 
perception pertaining to the practices of online learning 
internationally. 

2.2 Instrument and Samples 

In order to collect required empirical data and answer 
above said questions, the study designs and conducts an 
international online PBL program for K-l 2 schools hosted on 
a dedicated international online collaborative learning 
web site, APEC Cyber Academy (ACA, http://linc.hinet. 
net/apec/). The APEC Cyber Academy is an International 
Virtual Learning Environment for K-l 2 students to interact 
and collaborate on projects following the principles of 
social constructivism and self-regulated learning. The main 
objectives of ACA are: 

• Providing a networked learning environment that 
follows the design principles of PICI to facilitate 
interaction for learning: 

• Utilizing state-of-the-art technology to assist learning 
and assessment; 

• Applying the pedagogical principles of collaborative 
learning into the design of online activities: 

• Fostering international friendship among K-l 2 learners 
through online collaboration and computer-mediated 
communication; and 
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• Acquiring ICT (information and communication 
technology) skills through project-based learning, 

The ACA is founded on constructivist and self-regulated 
learning theories. The two theories complement each other 
well in fostering learner-centered learning strategy. While 
constructivism emphasizes that knowledge is co¬ 
constructed with peers or experts and through immersion in 
a social context, self-regulated learning places a strong 
emphasis on cultivating an individual learner's ability to be 
an autonomous learner. As such, ACA serves as a venue for 
implementing innovative pedagogy that nurture problem¬ 
solving, creativity, critical thinking, and collaborative 
learning skills (Lin, Chou, & Bagley, 2007). 

The international online PBL program of the study allows 
students and teachers to form teams online and conduct 
their own PBL projects in global context. The program was 
publicized through social networks and it attracted a total 
of 29 teams with 163 high school students from Philippines, 
Romania, South Korea, and Taiwan. During the five-month 
duration of the program in 2014, students formed small 
teams under the supervision of their teachers to participate 
in the program. The PBL program asks participants to design 
and carry out their own projects respectively in a team- 
based fashion. Participants are free to incorporate any 
media to create digital artifacts about their projects 
following the project-based learning guidelines set up by 
the program. The final artifacts of projects, in the form of 
either videos or web pages, were uploaded to ACA to allow 
participants to conduct peer evaluation and provide 
feedback to each other. Figure 2 depicts a showcase of 
artifact from participants of the program. 

In addition to investigating the patterns over community of 
practice with quantitative data recorded in system log file, 


Figure 2. A Showcase of Students Artifact 


ultimately the study also explores students and teachers 
perceptions of PBL regarding the fertilization of the 21 st 
century skills with surveys and observations. The study 
attempts to find out whether PBL could boost the 21 st 
century skills on high school students or not. 

2.3 Objectives and Questions 

The study is aiming to explore new frontiers in the field of 
innovative learning practices. There are three unequivocal 
phases regarding the objectives and questions of the 
study. (1) Is there a pattern over online learning behaviors 
among high school students? Does the pattern align with 
the proposed theoretical framework? (2) Do students 
behave differently across countries online? (3) Could 
international online PBL program promote the 21 st century 
skills among high school students? Lin and Ma (2011) once 
claimed that the best practice of PBL is in networked 
learning environments with supports of learning 
community. The answers of three questions of the study 
might further verify the claim. 

3. Results Pertaining to Community of Practice 

ACA system log file provides six variables pertaining to the 
study goals on learning community of practice. The six 
variables are Frequency of Login (FL), Time Online (TO) in 
Minutes, Frequency of Rating on Artifacts Committed 
(FORC), Frequency of Commentary on Artifacts 
Committed (FOCC), Frequency of Rating on Artifacts 
Received (FORR), and Frequency of Commentary on 
Artifacts Received (FOCR). Table 1 presents a glance at 
these six variables and their values in descriptive statistic 
measures in team basis. 

In the program, each participant is only allowed to provide 
rating once for each artifact. Flence the total values of 
FORC and FORR are identical. As to the total values of 
lentary variables, which are FOCC and FOCR, they 


Variables 

Total 

N 

Mean 

SD 

FL 

2071 

29 

71.4 

45.7 

TO 

6849 

29 

236.2 

183.1 

FORC 

399 

29 

13.8 

24.4 

FOCC 

207 

29 

7.1 

11.3 

FORR 

399 

29 

13.8 

8.0 

FOCR 

95 

29 

3.3 

4.0 


Table 1. Six Variables in the Study 
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are not identical due to the fact that the frequencies of 
expository replying committed by the owners of artifacts 
are excluded from the FOCR. Although ACA system does 
not confine the frequency of making comments on every 
artifact to each participant, both the values of FOCC and 
FOCR are smaller than their counter parts in rating artifacts, 
Meanwhile, the participation or engagement of 
participants in the program is quite high based on both the 
values of FL and TO variables, 

These six variables could be further categorized and 
corresponded to different levels of immersion in the 
aspects of learning community of practice based on the 
theoretical framework of the study. Both FL and TO variables 
together correspond to the observing level in the 
immersion hierarchy. The variable regarding with providing 
rating scores to artifacts, which is FORC, represents the 
participatory level. The FOCC variable, which is defined as 
contributing expository comments to artifacts, definitely 
could represent the contributory level. As to the next level, 
which is the collective level, it might associate with both 
FORR and FOCR in that these variables contains the nature 
of group discussion and exchange of the opinions 
regarding a specific artifact. Since the variable FORR is only 
a mirror or reflection of variable FORC in the sense of group 
performance, however, it will not involve in the discussion 
about the group immersion levels of community of 
practice. The upmost level of the immersion hierarchy, 
which is the pervasive level, could be presented by both 
the ratio of values between contributory level and 
participatory level and the ratio of values between 
collective level and participatory level. 

Based on the above definition of the five immersion levels 
and the data in Table 1, most of the students in the study 
reach only the contributory level. With ratios of 0.52 and 
0.24 respectively in pervasive level, it is quite obvious that 
only few students reach the pervasive level in the 
immersion hierarchy of community of practice. 

There are several more questions that are prominent to 
search for answers in terms of variation among countries 
pertaining to community of practice. Some of the 
questions are given by, 

(1) Is there a difference of performance on the level of 


observing between students of the four countries? 

(2) Is there a difference of performance on the level of 
participatory between students of the four countries? 

(3) Is there a difference of performance on the level of 
contributory between students of the four countries? 

(4) Is there a difference of performance on the level of 
collective between students of the four countries? 

As to the comparison over the performance on pervasive 
level between students of the four countries, it is omitted 
here due to the fact that few of the participants reach the 
upmost level of immersion hierarchy. Tables 2-5 are data 
related to these four questions in descriptive statistics 
fashion respectively. 

Due to the limited number of teams participated in the 
study, the Kruskal-Wallis analysis, instead of ANOVA, is utilized 
to compare the differences among four countries on the 
six variables. Table 6 summarizes the results of the data 
analysis. 

3. 11s there a difference of performance on the observing 
level between students of the four countries? 

When using data in Table 2 to compare means of FL and TO 
variables between the four countries with Kruskal-Wallis 
analysis, it turns out that students from both Taiwan and 
Philippines have higher frequency of visiting ACA significantly 
(p<0.01) than the students from both Korea and Romania. 
Plowever, there are no significant differences on time staying 
on ACA among students of four countries (p>0.1), although 
both Taiwan and Philippines also have longer duration of time 
online than the other two countries. 

Flung et al. (2012) argued that FL is a better indicator of 
students' engagement and performance online than is the 
TO. In this regard, students from both Philippines and Taiwan 
in the study are better than students from Romania and 
Korea significantly in terms of community of practice in the 





FL 




TO 


Country 

Total 

N 

Mean 

SD 

Total 

N 

Mean 

SD 

South Korea 

106 

3 

35.3 

7.8 

443 

3 

147.7 

110.8 

Philippines 

940 

8 

117.5 

40.7 

1991 

8 

248.9 

80.6 

Romania 

248 

7 

35.4 

23.7 

1469 

7 

209.9 

23.7 

Taiwan 

777 

11 

70.6 

36.8 

2946 

11 

267.8 

286.4 


Table 2. Performance on Observing Level 
between the Four Countries 
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observing level. 

3.2 Is there a difference of performance on participatory 
level between students of the four countries? 

The study defines participation in the community as 
frequencies that participants of the program are providing 
rating score to the artifacts. When using data in Table 3 to 
compare the mean of FORC variable between the four 
countries with Kruskal-Wallis analysis, results in Table 5 reveal 
again that students from Philippines and Taiwan perform 
better in the participatory level significantly (p<0.0T) than 
the other two countries. 

3.3 Is there a difference of performance on contributory 
level between students of the four countries? 

Writing or providing expository comments on artifacts 
submitted by peers in the community definitely is a 
contribution to the knowledge pool. The study compares 
the mean of FOCC variable in Table 4 between the four 
countries. The results in Table 6 indicate that performance 
of students from Korea and Romania are in the bottom and 
students of Philippines and Taiwan are on the top 
significantly (p<0.05). 

It is assumed that providing feedbacks with score is easier 
than with making comments in peer evaluation. After 
comparing means of FORC and FOCC variables with t-test 
for paired samples, it turns out that there is no significant 
difference here (t = -1.49, df = 28, p>0.1). The result 
implies not only that the common assumption about the 
feedbacks with score or comment is not correct, but also 
that students in the program reach contributory level of the 


Country 

Total 

N 

FORC 

Mean 

SD 

South Korea 

0 

3 


0 

0 

Philippines 

81 

8 


10.1 

11.4 

Romania 

0 

7 


0 

0 

Taiwan 

318 

11 


28.9 

33.5 

Table 3. Performance on Participatory Level 



between the Four Countries 



Country 

Total 

N 

FOCC 

Mean 

SD 

South Korea 

0 

3 


0 

0 

Philippines 

121 

8 


15.1 

17.5 

Romania 

1 

7 


0.1 

0.4 

Taiwan 

85 

11 


7.7 

6.3 


Table 4. Performance on Contributory Level 
between the Four Countries 


Country 

Total 

N 

FORR 

Mean 

SD 

Total 

N 

FOCR 

Mean 

SD 

South Korea 

3 

3 

1.0 

1.0 

9 

3 

3.0 

2.7 

Philippines 

124 

8 

15.5 

10.3 

16 

8 

2.0 

2.8 

Romania 

139 

7 

19.9 

3.1 

5 

7 

0.0 

0.5 

Taiwan 

133 

11 

12.1 

4.3 

62 

11 

5.9 

4.9 


Table 5. Performance on Collective Level 
between the Four Countries 


immersion hierarchy. 

3.4 Is there a difference of performance on collective 
level between students of the four countries? 

When a subject or object in networked learning 
environments could ignite members of the community to 
input and share their opinions about it together, which 
signifies a sign of its strength or quality on pouring in the 
collective wisdom. In this regard, both FORR and FOCR 
variables together could represent the performance on 
the collective level in the immersion hierarchy of the study. 

After analyzing FORR and FOCR variables presented in 
Table 5 with Kruskal-Wallis analysis respectively, results in 
Table 6 indicate that there is a difference between the four 
countries in FORR. As to the FOCR variable, the difference of 
performance between the four countries is not significant. 
When reviewing the data presented in Table 5 deliberately 
with the results of the statistical analysis in mind, although 
the performance of students from Philippines and Taiwan 
are as steady as in the previous levels, students from 
Romania is a little bit belter off on the level of collective, All 
of these confounding results is an evidence that students in 
the program do not reach the collective level of the 
immersion hierarchy. 

4. Results of Survey Pertaining to Building of 21 st Century 
Skills 

With regard to the building of the 21 st century skills, the study 
employs a mixed method research that utilized data from 
surveys, focus groups, and observations. The surveys that 


Variables x 2 df P 


FL 

15.3 

3 

0.002 

TO 

2.7 

3 

0.436 

FORC 

13.5 

3 

0.004 

FOCC 

10.6 

3 

0.014 

FORR 

13.3 

3 

0.004 

FOCR 

3.9 

3 

0.268 


Table 6. Results of Kruskal-Wallis Analysis 
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address the research goals were posted online at the end 
of the international PBL program for teams over all four 
countries. Three focus groups and observations were 
conducted with teams at the classroom sites toward the 
end of the program. A total of 11 7 student participants 
(response rate: 72%) and 10 teachers (respond rate: 35%) 
completed the online survey. 

The data from open-ended parts of the surveys, classroom 
observation notes, and focus group transcripts were 
collected and analyzed using Nvivo, a qualitative analysis 
software program. The results can be summarized as 
followings: 

4 .1 Communication & Collaboration 

Students found that the processes of working with group 
members and engaging in online peer reviews have 
helped them to communicate and work better with each 
other. 

4.2 Creativity 

The multimedia production process has brought forth 
many creative project ideas that were not possible in 
regular class activities. 

4.3 Critical Thinking 

The creation of the projects has helped the students to 
expand views about the themes of the projects. The 
process has also enhanced their critical thinking skills while 
researching and analyzing data towards drawing the 
conclusions of the projects. 

4.4 Engagement 

Students found creating digital projects for international 
partners a fun activity to engage in. 

4.5 Improved Technological Skills 

Teachers found that students did demonstrate 
improvements in utilizing digital media through the 
projects. The digital literacy and media skills can also aid 
learning in other courses. For example, students learned 
how to create digital artifacts and they can also transfer the 
skills to other courses. 

4.6 Increased Sense of Responsibility 

Students became more responsible in completing their 
own projects and working with others, 


4.7 Better Time Management 

Students were able to deal with deadline better because if 
they do not get their projects done, the international 
partners would not have any artifacts to comment on. 

The findings from the study have shown many benefits of 
PBL, especially in an international context. In summary, 
student and teacher perceptions of international PBL have 
shown increased communication, collaboration, 
creativity, and critical thinking skills throughout the process. 
Conclusion and Recommendations 

Results of the study both confirm the sequential and 
hierarchical nature of the asserted theoretical framework 
pertaining to community of practice and reveal that 
students perform differently among countries in virtual 
learning space. Based on the consolidate statistic data, 
the study also realizes that community of practice in the 
online PBL program has reached the contributory level. The 
confounding results when comparing the difference of 
performance between the four countries on the collective 
level also imply that the community of practice in the 
program did not reach this level. Under the circumstances 
of the current schooling environments around the globe, 
especially in Asian countries which are notorious with 
college entrance exams combing, performance of the 
community of practice in the online PBL program 
implemented by the study is above expectation and far 
more than admirable. Meanwhile, the study also proves 
that PBL is beneficial to fostering the 21 st century skills on 
high school students in a global context. 

Overall, due to the limitation of the amount of participants 
across countries results of the study ore likely less 
conceivable, especially in the area of community of 
practice with quantitative measures. As a result, the study 
only provides a glimpse at the patterns of online learning 
behaviors among high school students in general and in a 
global context in particular. Ultimately, findings of the study 
are solid cornerstones and references for further studies in 
the frontier of new schooling with information technology. 
Above all, the study does prove that PBL provides a unique 
opportunity to help students practice critical thinking, 
collaboration, communication, problem-solving, and 
creativity. It is therefore highly recommended to schools to 
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adopt PBL when facing challenges of creating innovative 
pedagogies for the generation of digital natives. The other 
significant finding of the study is that both PBL and 
community of practice is a fantastic blending combination 
and they are tightly dependent on each other. The finding 
reconfirmed the claim made by Lin and Ma (2011). While 
practicing PBL pedagogy, therefore, the findings of the 
study suggests that the best practice of PBL is implemented 
or conducted in networked learning environments in a 
global context with vibrant community of practice, since it 
will result in deeper learning and engaged, self-directed 
learners. 
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